Assessing the Need for a National Disability Survey: Final Report. C. Limitations of Existing Data on Disability


Although extensive information pertaining to disability is collected through national surveys and program administrative data, the existing sources of data on disability have a number of shortcomings that limit their usefulness. Gaps and limitations of available disability-related data were identified by the staff of the state and federal agencies that provided input to this study. Table II.4 lists specific questions they noted as being unanswered due to disability data limitations. These cover a wide range of topics that we have organized into several broad topics areas: program participation, education, employment, housing and independence, quality of life, health, and health care.3

The inability to address the numerous unanswered questions identified by state and federal staff stems from a variety of limitations of existing disability data. These limitations relate to the manner in which disability is measured, small sample sizes, data accessibility, the quality of collected data, poor coverage of certain disability-related topic areas, and the frequency of data collection. In the sections that follow, we discuss the nature of these limitations.

TABLE II.4. Unanswered Questions Due to Data Limitations Identified by State and Federal Agencies
Program Participation
  1. How many people with disabilities participate in multiple programs and what are their characteristics?
  2. Is there duplication or fragmentation of services for those participating in multiple federal disability programs?
  3. What is the cost-effectiveness of services for which consumers use comparable benefits?
  4. What happens to persons with disabilities after they leave public assistance programs?
  5. What are the incentives that affect individuals’ decisions to either apply or not apply for DI and SSI benefits?
  6. What are the major referral sources for Social Security disability programs?
  7. How many people would meet SSA’s definition of disability if they were not working and/or chose to apply?
  8. How to do community rehabilitation providers rate with respect to the impact of service delivery after accounting for disability severity?
  9. How can researchers model the behavior of the nonelderly Medicaid disabled population?
  10. Are food assistance programs meeting the needs of disabled households?
  11. What are the types and numbers of customers with disabilities being served by the One-Stop system?
  12. How effective are state and federal benefit counseling services?
  13. What is the total federal compensation paid to disabled veterans?
  1. How does educational performance vary by type of disability?
  2. What services and accommodations are used by students with disabilities and how effective are they?
  3. What are the characteristics of students with disabilities?
  4. What happens to children who receive special education services but do not receive adult developmental services?
  5. How do people with disabilities make transitions from education to employment?
  6. What effect has rehabilitation had on drop-out and graduation rates?
  1. What are the characteristics of working people with disabilities?
  2. What is driving the lower labor force participation rates among people with disabilities?
  3. What enhances return to work and long-term job retention following return to work?
  4. What are the quarterly employment and earnings patterns of SSDI/SSI applicants and beneficiaries?
  5. What are the quality and level of employment and how do they compare to desired employment?
  6. What are the labor market outcomes following participation in education and training programs?
  7. How are period of military service and race related to disability among veterans?
Housing and Independence
  1. What kinds of assistance and services are needed for independent living in the community?
  2. What are the income level and housing needs of persons with disabilities living in noninstitutional group quarters?
  3. How do young people with disabilities make arrangements for housing following transition to adulthood?
  4. What is the relationship between housing and disability?
Quality of Life
  1. What is the relationship between disabilities and household food insecurity?
  2. What are the consumption and expenditure patterns of persons with disabilities?
  3. What are the outcomes and quality of life for people with disabilities?
Health Status
  1. How do morbidity and mortality relate to disability?
  2. What are the state-level prevalence rates of specific conditions?
  3. How prevalent are comorbidities among people with disabilities?
  4. How prevalent are intellectual disabilities and how do they affect overall health status?
  5. How prevalent are mental health limitations, and are limitations underreported?
  6. How does disability policy affect the dynamics of mental illness?
  7. What are the dynamics of disability (spells, duration)?
  8. What are the prevalence rates of disability for foster youth?
  9. What are the oral health status and immunization rates among people with disabilities?
Informal and Formal Health Care
  1. What are the characteristics of co-residing caregivers and how does this care affect outcomes?
  2. To what extent do people with disabilities receive informal assistance with specific activities?
  3. How do long-term care programs and policies affect families and their role in care?
  4. What are the characteristics of the provider networks of working people with disabilities?
  5. What are the health care needs of the employed people with disabilities (rather than “utilization”)?
  6. What functional and other factors are related to potentially avoidable hospitalization?
  7. What are the patterns of morbidity and utilization rates among states with high penetration of managed care patients?
  8. What is the medication usage of people with disabilities?
  9. What are the health care usage patterns of individuals who transition across systems and environments?
  10. How does continuity of care over time affect health outcomes and functioning?
Health Care Costs and Quality
  1. What is the cost-effectiveness of care in programs that serve dual-eligibles?
  2. Should capitation rates to Medicare Advantage plans be adjusted to reflect the functional status of their enrollees?
  3. How do costs and utilization in Medicaid vary by SSDI/SSI status and length of time enrolled?
  4. How can access to and satisfaction with care be improved for the Medicare population with disabilities under age 65?
  5. How do people with disabilities’ experiences in and satisfaction with Medicaid home and community-based service programs vary across states?
  6. What are the experiences of people with disabilities in managed care?
  7. What is the quality of care of programs that serve dual-eligibles?
  8. How can populations of people with certain disabilities be identified for specific quality measurement?

1. Disability Measurement

The health and functional measures used to identify people with disabilities vary markedly in concept, detail, and quality across surveys. The lack of consistency across surveys leads to widely different prevalence measures and an inability to identify a common target population for which information across surveys might be used to develop a comprehensive picture of people with disabilities and their circumstances. Staff of many agencies pointed out that comparisons across surveys and populations are difficult at the national level, and even more so when attempting to crosswalk from national data to state-level data. As one participant noted, “While there has been improvement in the standardization of disability screeners, there is still a lack of consistency in disability questions among surveys.”

Except for the surveys that focus on health, most do not capture in-depth information on the health issues and functional limitations experienced by people with disabilities. Many surveys ask only a small sequence of questions related to disability, such as whether the survey respondent faces a work disability or has a limitation in one of six common functional categories (hearing, vision, mobility, cognition, self-care, and independent living). State and federal agencies identified many other areas for which detailed disability information was lacking, including age of onset, disability permanence, severity, general health status, functional status, activity limitations, and underlying medical conditions. Further, difficulty identifying people considered to have disabilities under varying programmatic definitions of disability was noted. For example, staff of one agency commented, “The current six questions in the ACS and CPS do not contain the specificity that we need to identify individuals who might be eligible for our program.”

Measures corresponding to mental, emotional, cognitive, and intellectual disabilities were identified as being particularly poor. One agency noted that “People with intellectual disabilities [are] not routinely included in surveys, due to the issues of consent, mode of survey (telephone, in-person, mail, Internet), [and] question development.” Another agency expressed concern over not being able to estimate the prevalence of mental disabilities and substance use disorders.

One FAEP member also pointed out that surveys with the same disability questions sometimes produce prevalence estimates that are different. This might reflect differences in the context of the questions or differences in survey methodologies.

2. Sample Sizesand Subgroups

The federal and state agencies that provided input to this study identified a number of subgroups for which disability-related data were lacking, primarily because of small sample sizes in the national surveys, but in some cases because these groups simply cannot be identified in existing national surveys. These subgroups include the three primary age groups (youth, working-age persons, and seniors), veterans, homeless people, people with specific health conditions or types of disabilities, various demographic subgroups, people who are not in the labor force, participants in public programs, and those receiving or providing formal and informal care. Table II.5 lists the subpopulations that were specifically noted. In some cases agencies noted specific unanswered questions for a subpopulation; in such cases the questions are listed in Table II.5. Otherwise, only the subpopulation is listed.

Small numbers of people with disabilities in the large, general population surveys limit the ability to examine subgroups of people with disabilities. For example, subgroup analyses by specific types of disabilities, and those focused on current and potential program participants, cannot be conducted in many cases. One agency respondent suggested that small sample sizes might be a consequence of the inability of the currently used measures to capture people with psychiatric and intellectual disabilities. Many disabling health conditions are relatively uncommon and small sample sizes make identifying people with specific types of disabilities difficult. Staff of another agency noted that small sample size issues are most severe for those under age 65 because disability prevalence is relatively lower for this population.

Analysis of residents of specific regions, states, or substate geographic areas is also limited due to small sample sizes. As one respondent noted, there are “insufficient sample sizes to conduct state-specific estimates on people with disabilities.” Others also noted that state or other geographic-level analyses of people with specific conditions, functional limitations, and in various programs such as Temporary Assistance for Needy Families and Medicaid are not possible.

TABLE II.5. Subgroups for Which Disability Data Are Lacking Identified by State and Federal Agencies
Demographic Subgroups
  1. Distribution of those with psychiatric disabilities by age and gender
  2. Race/ethnicity and male/female
  3. To what extent do the populations of persons with disabilities overlap with minorities and low-income populations? How are these populations geographically distributed?
  4. More general information on people in the following groups:
    • American Indians
    • Asian Americans
Age Groups
Children and Youth
  1. Children (adequate sample sizes, disability measures, and family characteristics)
  2. Institutionalized children with disabilities--more information about their health status
  3. Children with parents with disabilities
  4. Youth with disabilities transitioning from school to work
  5. Students with intellectual disabilities and postsecondary educational outcomes
Working-Age Individuals
  1. Nonelderly persons with disabilities
  2. Dual-eligibles under age 65 with serious mental illness, relevant for states pursuing the Health Home option under the Affordable Care Act
  3. Ages 18-64: participation in managed long-term care, Medicaid Buy-in, and competitive employment
  4. What are the health trends of those under age 65? (to predict future outcomes when over age 65)
  1. Elderly persons with disabilities and specific health issues that would benefit by having services brought to them
  2. People with disabilities ages 55-70 and how they would respond to changes in SSA and Medicaid eligibility ages
  3. Older persons residing in nursing homes and/or other institutions who could work
  1. Veterans--homelessness, access to Social Security and SSI benefits, family economic characteristics
  2. Do VA benefits help veterans improve their health and well-being and return to work, among other outcomes?
  3. How and when do recent veterans seek benefit services from the VA?
Homeless Individuals
  1. Homeless SSI/SSDI applicants and beneficiaries--income, resources, access to Social Security and SSI benefits
  2. Distribution of those with mental health disabilities by homelessness history
Specific Types of Disabilities
  1. Analyze services for individuals with disabilities by disability type
  2. People with AIDS who would benefit by having services brought to them
  3. Distribution of those with psychiatric disabilities by major diagnostic category
  4. Analyze services for individuals with co-occurring medical conditions
  5. Participation in managed long-term care, Medicaid Buy-in, and competitive employment by type of disability
  6. Specific limitations among race and demographic subgroups
  7. One-Stop customers with nonvisible disabilities (mental illness, learning disabilities)
  8. More general information on people with the following conditions:
    • Spinabifida, nationally and at the state level
    • Paralysis and limb loss, nationally and at the state level
    • Deafness or hearing loss
    • Mental illness
    • Behavioral health issues
    • Intellectual disabilities
    • Acquired brain injury
Employment Subgroups
  1. Competitively employed individuals with disabilities
  2. Institutionalized people with disabilities who could work
Program Participants
  1. Disability programs and various populations they serve
  2. More information about Medicaid Buy-In participants
  3. More information about those participating in Medicaid home and community-based services waiver programs
  4. SSDI and SSI beneficiaries
    • What access problems do SSDI beneficiaries in the 24-month waiting period experience and do they affect health outcomes?
    • How do allowed and denied disability applicants differ in terms of their characteristics?
  5. What are the characteristics of and outcomes for beneficiaries who work?
  6. What are the characteristics of and outcomes for beneficiaries who leave the rolls due to medical improvement?
  7. What are the outcomes for and number of beneficiaries enrolled in other programs?
Long-Term Care and Informal Care Providers and Users
  1. More information about caregivers
  2. Services provided by nonpaid caregivers
  3. More information about the family of respondents in disability surveys
  4. Capturing the characteristics, medical expenditures, and utilization for persons residing in institutional settings
  5. More information about long-term care settings, including receipt of care at home
  6. More information about managed long-term care users

3. Administrative Data Access

Federal and state agency respondents acknowledged that use of survey data could be bolstered by linkages with administrative data collected by the programs that serve people with disabilities. However, data access issues and difficulties associated with performing such linkages were identified as hindering that process. Administrative data are often hard to access due to legal, privacy, and security concerns,and data-sharing agreements are sometimes difficult to obtain. For those who were able to obtain access to necessary data, linking the data across sources proved to be problematic for various reasons. For example, FAEP members noted working at agencies that did not have adequate staff, time, and resources to manage data linkages. One state agency noted that states lack technical assistance in accessing and linking federal data. A federal agency noted that these barriers are even greater for nongovernmental researchers.

Although administrative data sources appear to be used extensively by federal agencies, respondents noted a number of limitations associated with these data:

  • Difficulty accessing administrative data from other agencies. Several respondents indicated that obtaining data-sharing agreements across agencies was sometimes difficult and time-consuming.

  • Difficulty with and lack of resources available for linking data. Several respondents noted that linking data is a technically demanding and resource-intensive process. Even when access is granted, adequate documentation, resources, and technical expertise are required to successfully link and analyze the data.

  • Timing of the data. A few respondents also mentioned that the quarterly or annual nature of some administrative data (for example, earnings data) make them much less useful than if the data were collected and reported more frequently. In addition, administrative data made available for use more widely are often released with a considerable time lag.

Another potential problem with attempting to link administrative data to survey data is that this is primarily done through the collection of Social Security Numbers (SSNs) from survey respondents; respondents may refuse to provide this sensitive information due to identity theft or other concerns. Even among those willing to provide SSNs for themselves or others in their households, missing or incomplete information may be an issue. High refusal rates or incomplete SSN information will affect the quality of survey-administrative data linkage. As an example, the NHIS files linked with SSA administrative data that are maintained by NCHS have match rates that declined fairly steadily over time, from 75 percent in 1994 to 46 percent in 2005 (NCHS 2009). The low match rates in the later years will certainly affect the perceived quality of the data by researchers, and require those using the data to conduct analyses to assess the extent to which “nonmatching” bias might affect the findings of their studies. The Census Bureau has developed a method to link data that does not require participants to provide their SSNs (O’Hara 2007), which we describe in the next chapter.

4. Data Quality

Federal and state agency respondents cited data quality concerns in conducting disability-related research. Several listed specific datasets for which they believed the data were inaccurate or inconsistent. As a result, one agency noted that they had deemed certain data sources as “insufficiently reliable for specific purposes.” Data was thought to be particularly poor for age and gender subcategories, people with psychiatric disabilities, veterans, those participating in Medicaid managed care, and people receiving employment support services. It was postulated that proxy responses and stigma related to disability might contribute to poor data quality.

During the FAEP meeting, the issue of data quality was raised with respect to respondent reports of program participation. One meeting participant noted that his agency had found self-reports of program participation to be inaccurate when compared against administrative data. The lack of administrative data linkages across multiple programs and with survey data inhibits the ability to assess program participation, duplication, and adequacy of the services and benefits provided by federal programs.

5. Survey Content

A number of subject areas were identified as being inadequately addressed for people with disabilities. Subject areas deemed lacking include consumption, long-term supports, family structure, and informal care. Data also were perceived to be lacking for people who are similar to those enrolled in programs but who are not enrolled, as well as for people who have exited a program. For example, one agency stated “We have administrative data to track persons with disabilities when they are on Medicaid; however, we do not have data when they are not on Medicaid. This makes it impossible, for example, to track persons’ well-being after exiting from Medicaid.” Other examples of inadequate survey content were reflected in the unanswered questions and subpopulations of interest identified in Table II.4 and Table II.5.

During the FAEP meeting, participants elaborated on particular subject areas raised as being inadequately addressed for people with disabilities. These include:

  • Program Participation and Adequacy of Assistance. Participants noted a lack of information about the adequacy of income assistance programs for people with disabilities, and a lack of measures of material hardship, quality of life, and other outcomes that would allow an assessment of whether programs are meeting their objectives. Participants noted the difficulty of identifying people with disabilities participating in multiple public programs, and so issues related to adequacy, duplication, and fragmentation of services cannot be addressed.

  • Community Living. Inadequate information about unmet service and support needs of people with disabilities was discussed in the context of the demand for accessible housing and more generally, in terms of what is needed to help those with significant disabilities live in the community. While there is some information at the national level about housing accessibility and services to support community living, such information is absent at the local level. It was noted that information is lacking about the types of services that people with significant disabilities need to live in the community, or the availability of those services in local (urban and rural) areas. For example, supports that are either formally or informally provided, including personal care, assistance with household maintenance, accessible housing, accessible and reliable transportation, home modifications, assistive technology, and particular types of job accommodations.

  • Employment. Information about how individuals with significant disabilities are able to stay employed is limited. Members also had an interest in having more information about specific employment barriers and the extent to which individuals with disabilities believe that they are employed at their full potential. Other employment issues discussed related to underemployment among those with postsecondary education, and how the employment of and use of services by people with disabilities is affected by economic changes.

  • Expenditures. FAEP members noted a lack of information about the consumption and expenditure patterns of people with disabilities, primarily because of the lack of disability measures in the CE. Information on expenditures might be used to infer service needs and use in order to address some of the unanswered questions noted in the three preceding topic areas.

  • Health Care and the Affordable Care Act. Members noted that we lack information about the experiences of people with significant disabilities enrolled in employer-sponsored health insurance, and so are unable to infer the extent to which new sources of coverage implemented under the Affordable Care Act will meet the needs of people with disabilities. More generally, it was noted that information on health care access, quality, and unmet needs for people with disabilities is severely limited by existing data.

6. Timingand Trends

Federal and state agency respondents raised several issues related to survey data frequency and timing. One issue relates to the timeliness of available data; one respondent suggested that for programmatic purposes, old data were no longer helpful or relevant. Another issue concerned the lack of current trend data as some surveys transition to new measures of disability. One agency stated that “new instruments… are too new to provide information on trends.” Finally, several responses indicated that the lack of longitudinal data is an issue. Where these data exist, one respondent noted that longitudinal datasets fail to “reasonably identify persons with disabilities.” As some disabilities might be temporary or episodic, their effects are sensitive to the timing of the survey. FAEP meeting participants also noted that existing data do not provide a lifetime perspective on disability. Age and population-specific surveys often provide detailed information, but only for a relatively short window of time or only for older individuals. Accordingly, it is difficult to understand the dynamics of disability onset at younger ages and changes in severity and functioning over time.

View full report


"NatlDS.pdf" (pdf, 1.5Mb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®