Assessing the Need for a National Disability Survey: Final Report. Appendix A. Federal and State Agency Feedback on Disability Data Needs and Limitations


A. Questions for Federal and State Disability Policy Stakeholders

Current and Past Disability Data Needs and Uses

  1. What are the disability-related policy/program/research questions that are important to your agency?

  2. Does your agency have specific legislative, regulatory, or other needs for disability data? If so, what are they?

  3. What sources of data have been used to address the above disability-related questions and information needs?

    • What, if any, survey data does your agency use?
    • What, if any, administrative data does your agency use?
  4. Have you used merged survey and administrative data for addressing questions like those noted above and if so, how have you used these data?

Addressing Disability-Related Data Gaps and Limitations

  1. Are there any disability-related program or policy questions that are important to your agency/program(s) but which you cannot address due to data limitations? If so, what are those questions?

  2. What is the nature of the data limitations that prevent your agency from addressing the above questions (e.g., limitations related to lack of access to administrative data matched to survey data or to data from other agencies, small survey sample sizes, inadequate survey content,infrequent data collection)?

  3. What strategies, if any, have you used in the past to address such limitations?

  4. Are there any disability-related studies or initiatives that your agency would like to conduct, but have not done so due to data limitations? If so, please describe.

  5. Are there any subpopulations of interest to your agency (e.g., individuals in specific age groups, with specific types of disabilities, residing in particular institutional settings), but about which little is known because of data limitations? If so, please describe the group and the kinds of information you wish were available.

  6. What kinds of data would be most useful to address the important unanswered questions or otherwise help your agency in disability-related program and policy activities?

  7. Is your agency planning or in the process of developing any new disability-related survey or administrative data collection efforts?

  8. Please provide any additional thoughts or comments you have related to the nature and availability of data to address disability-related program and policy issues.

B. List of Agencies That Provided Input

  1. Administration on Aging

  2. Agency for Healthcare Research and Quality, Center for Financing, Access, and Cost Trends

  3. Bureau of Labor Statistics

  4. Census Bureau, Housing and Household Economic Statistics Division

  5. Center for Health Care Strategies

  6. Centers for Disease Control and Prevention, National Center on Birth Defects and Developmental Disabilities, Division of Human Development and Disability

  7. Centers for Medicare and Medicaid Services

  8. Congressional Budget Office, Budget Analysis Division, Long-Term Modeling Group and the Income Security Unit

  9. Congressional Budget Office, National Security Division

  10. Connecticut Department of Rehabilitation Services

  11. Maryland Rehabilitation Services

  12. National Center for Health Statistics

  13. National Council on Disability

  14. National Institute of Health, National Institute on Aging

  15. National Institute of Mental Health

  16. Substance Abuse and Mental Health Services Administration

  17. University of Massachusetts, Disability Health and Employment Policy Group within the Center for Health Policy and Research & Shriver Center

  18. U.S. Department of Agriculture, Economic Research Service

  19. U.S. Department of Education, National Institute on Disability and Rehabilitation Research

  20. U.S. Department of Education, Office of Safe and Drug-Free Schools

  21. U.S. Department of Education, Office of Special Education Programs

  22. U.S. Department of Education, Office of the Undersecretary

  23. U.S. Department of Education, Rehabilitation Services Administration

  24. U.S. Department of Health and Human Services, Immediate Office of the Secretary, Office on Disability

  25. U.S. Department of Housing and Urban Development, Office of Policy Development and Research

  26. U.S. Department of Labor, Employment and Training Administration

  27. U.S. Department of Labor, Office of Disability and Employment Policy

  28. U.S. Department of Transportation, External Civil Rights Program

  29. U.S. Government Accountability Office, Education, Workforce and Income Security

  30. U.S. Social Security Administration

  31. Wisconsin Department of Human Services, Office of Independence and Employment

C. Summary of FAEP Meeting Proceedings

The Federal Advisory Expert Panel (FAEP) meeting for the Assessing the Need for National Disability Survey project was held in May 2011. Prior to the meeting, input was solicited from 31 federal agencies and state organizations. This input covered the current disability data uses of federal and state agencies, unanswered question and disability-related data gaps, and agency-specific efforts to address data shortcomings. The purpose of the meeting was to: expand on the disability-related questions that were identified as being important to participants’ policy and program efforts that remain unaddressed because of data limitations; discuss the limitations of existing survey data that hinder their efforts; and discuss potential solutions to the limitations.

The meeting included three discussion sessions: (1) disability-related questions that remain unanswered due to data limitations, (2) limitations of existing data, and (3) options for improving disability-related data. Each of the three sessions began with a brief summary of the pre-meeting input provided by federal and state staff, followed by a discussion of additional topic areas not identified prior to the meeting and the importance of various suggestions. The sections that follow summarize the meeting discussion.

1. Disability-Related Questions that Remain Unanswered Due to Data Limitations

Prior to the meeting federal and state organizations were asked to complete a questionnaire regarding their experiences with disability-related data (see Section A of this Appendix). Two of the questions solicited information about unanswered disability-related program or policy questions and disability-related initiatives that have not been conducted due to data limitations. Responses included issues related to program participation, education, employment, housing/independence, quality of life, health status, and health care. In a separate question, respondents also indicated that data was lacking for several subgroups defined by age, other demographic characteristics, veterans, homeless individuals, specific types of disabilities, employment subgroups, program participants, long-term care users, and informal care providers and users.

FAEP members echoed many of the topics that had been identified prior to the meeting. Education, particularly postsecondary education, questions remain unanswered despite a general sense of the abundance of surveys containing information on disability and education. Even more questions about employment were discussed. In the pre-meeting input, health care was the topic for which the most unanswered questions were identified. During the meeting, health care issues were discussed again, with a focus on care quality and access. Below, we highlight specific topics of discussion.

Program Participation and Adequacy of Assistance. Throughout the discussion, unanswered questions about public programs were mentioned several times. Participants identified a lack of information about the adequacy of income assistance programs for people with disabilities (including veterans), and a lack of measures of material hardship, quality of life, and other outcomes (such as employment) that would allow an assessment of whether programs are meeting their objectives. At a more basic level, participants noted the difficulty of identifying people with disabilities participating in multiple public programs, and so issues related to adequacy, duplication, and fragmentation of services cannot be addressed. Members also noted an interest in understanding how program take-up rates change during an economic downturn and subsequent recovery.

Community Living. The issue of unmet service needs of people with disabilities was discussed in the context of the demand for accessible housing and more generally, in terms of what is needed to help those with significant disabilities live in the community. FAEP members noted that, while there is some information on the national level about housing accessibility and services to support community living, such information is needed at the local level. It was noted that we simply do not have information about the types of services that people with significant disabilities (i.e., those who need a nursing level of care) need to live in the community, or the availability of those services in local (urban and rural) areas. The inability of the American Community Survey (ACS) to provide local-level estimates of disability prevalence until the 3-5 year estimates become available (in 2012) was noted as contributing to the lack of data at the local level.

Employment. FAEP members noted a lack of information about how individuals with significant disabilities are able to stay employed. For example, what are those who are successfully employed doing to get their needs met? What types of programs and services (private or public) do they utilize? Members also had an interest in more information about specific employment barriers and the extent to which individuals with disabilities believe that they are employed at their full potential. Other employment issues discussed related to underemployment among those with postsecondary education, and how the employment of people with disabilities is affected by economic changes.

Expenditures. FAEP members noted a lack of information about the consumption and expenditure patterns of people with disabilities. If information on expenditures were available, it might be possible to infer service needs and use from these data to address some of the unanswered questions noted in the three preceding topic areas. A primary reason for the lack of expenditure information is the lack of disability measures in the Consumer Expenditure Survey.

Longitudinal Perspectives. FAEP members noted that existing data do not provide a lifetime perspective on disability. Age and population-specific surveys often provide detailed information, but only for a relatively short window of time or only for older individuals. Accordingly, it is difficult to understand the dynamics of disability onset at younger ages and changes in severity and functioning over time.

Health Care and the Affordable Care Act. Members noted that we lack information about the experiences of people with significant disabilities enrolled in employer-sponsored health insurance, and so, are unable to infer the extent to which new sources of coverage implemented under the Affordable Care Act will meet the needs of people with disabilities. More generally, it was noted that the ability to assess the health care access, quality, and unmet needs for people with disabilities is severely limited by existing data.

Discussion during the first portion of the meeting was focused on unanswered questions more so than inadequately-covered subpopulations. Attendees did reiterate calls for data on homeless individuals, those with behavioral or emotional disabilities, and those residing in group quarters.

2. Limitations of Existing Survey Data

Shortcomings of existing data that were identified by federal agencies and state organizations before the meeting include: poor and inconsistent measures of disability, small sample sizes, difficulty accessing and linking administrative data, poor data quality, inadequate survey content, and timing issues. During the meeting, all of these limitations were noted.

Disability Measurement. Some participants expressed the need for a standard, threshold measure of disability to be used across all federal surveys. There was some disagreement about the extent to which such a standard definition would be useful, however. One participant noted that even when using the same questions, different populations are identified because of differences in survey context, sampling, and administration. In addition, there is general disagreement about what constitutes disability. One participant shared news that the U.S. Department of Health and Human Services is supporting a requirement that the six-question series currently used to identify people with disabilities in the ACS (as well as the Current Population Survey and the American Housing Survey [AHS]) be included in all federally-sponsored surveys. Problems with these questions were noted, however. In particular, they likely fail to identify a large number of people with significant behavioral, emotional, and intellectual disabilities. Participants noted that developing a better understanding of who is captured by the six-item ACS measure and how it might be improved is critical. The evolution of the six-question sequence was questioned by one participant, and if promulgation of the measure is being advocated, this participant and others believed it important to understand what it is measuring and if it can be improved.

Other issues related to disability measurement that were discussed included the need to measure disability severity, developing a better understanding of the specific health conditions that are likely to result in disability, and incorporating the International Classification of Functioning, Disability, and Health (ICF) concepts into disability measures (both survey and clinical measures). It was also noted that, even if using a standard definition of disability, disability concepts need to be applied differently by major age group.

Data Quality. The issue of program participation was raised again during the discussion of data limitations. One participant noted that his agency had found self-reports of program participation to be inaccurate when compared against administrative data. The lack of administrative data linkages across multiple programs inhibits the ability to assess program participation, duplication, and adequacy.

Other issues were also raised by participants. Small sample sizes limit the ability to do subgroup analysis, particularly on people with specific disabilities or at the state and local levels. Lags in data availability make it difficult to produce timely reports. Participants also again noted that longitudinal disability-related data was lacking, especially long-term longitudinal data.

3. Options for Improving Disability-Related Data

In the background materials provided for the meeting, several options for improving disability data were described. These include: increasing awareness and use of existing data, improving disability measures, collecting longitudinal information, increasing survey and administrative data linkages, enhancing disability-related content, fielding periodic survey supplements, augmenting samples in existing surveys, conducting periodic surveys of subpopulations, and fielding a periodic national disability survey. Several of these options received recognition at the meeting, but little discussion. The meeting discussion generally focused on the topics below.

Increasing Access to Administrative Data. Options for improving access to administrative data described included: improving documentation for administrative data; facilitating data sharing, which might be done by having government agencies link administrative and survey data, then de-identify the data and make it publicly available; making administrative data available more timely; facilitating access to administrative data (for example, through the use of a contractor to administer data use agreements under a model similar to that used by the Centers for Medicare and Medicaid Services’ Research Data and Assistance Center); improving match rates between survey and administrative data; and increasing linkages between survey and administrative data (for example, doing so with the ACS).

Improving Disability Measurement. Participants discussed the development of a standard, reliable definition of disability that captures the appropriate population. There was no consensus regarding an appropriate definition, but many participants stressed the importance of developing the definition through a conceptual model that accounts for one’s environment, such as the ICF model of disability. It was also noted that the context of disability varies greatly with regard to age; separate disability definitions based on age (children, working-age, and elderly) may be more appropriate than one definition applied to all age groups. One participant was a strong proponent for disability being defined in terms of severity, as opposed to a threshold. This participant suggested that a four-level system of severity be implemented: (1) not limited, (2) somewhat limited, (3) limited, (4) completely unable to perform the given activity. Another suggestion was to add questions to the National Health Interview Survey (NHIS) that would allow for an assessment of the six-question disability measure and determine who is and is not being captured.

Other Options. Other suggestions offered by participants included:

  • Support more and longer-running longitudinal surveys, but many cited concerns with costs related to longitudinal data collection and low response rates. Using retrospective questions in cross-sectional surveys to elicit longitudinal-type information and increased linkages with administrative data were viewed as more practical options, although they have their own limitations.

  • Develop and disseminate information about existing data, in a manner similar to that done through the Aging Forum report on older Americans.

  • Find ways to make greater use of electronic health records or other encounter data to supplement existing data. Participants acknowledged that use of electronic records is not feasible at this point in time, but may be a potentially important source of data on disability in the future. It was also noted that, if we expect to be able to use these data for program and policy purposes, work needs to begin now, while the standards and content of electronic health records are still being developed, to ensure that the appropriate information is incorporated.

4. Summary

FAEP participants discussed a wide variety of topics reflecting the different data needs of their agencies. Notably absent from the open discussion of options was the suggestion to field a new national disability survey as a solution to the data shortcomings. When the group was asked to consider this option and discuss potential topics to be included if such a survey were to be conducted, the suggestions offered included the following:

  • Access to and quality of health care, including the specific services that are covered by insurance and the gaps/unmet needs.

  • Unmet service needs and the extent to which the existing system of supports fails people with disabilities in terms of supporting employment, community living, and independence.

  • Environmental and accessibility factors that affect disability and disability severity.

  • Subjective well-being, measures of quality of life, and measures of self-determination.

  • Measures of disability that could be used to assess and improve existing survey measures.

The lack of discussion of a national disability survey should not be interpreted to mean that participants were not interested in that option. The options discussed were raised primarily based on the discussion that occurred during the first session. In addition, the focus of the meeting was not on the options, but rather, on the policy and program issues that remain unaddressed due to data limitations. The discussion of options to address the data limitations was the focus of a Technical Advisory Group (TAG) meeting of academic researchers and representatives from government agencies involved in producing survey data that was held in June 2011 (see Appendix C for a summary of the TAG meeting).

View full report


"NatlDS.pdf" (pdf, 1.5Mb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®