Resources for Improving Demographic Data Quality

The resources provided in this section are designed to help those who are developing or implementing data collections to ensure that their approaches and content will result in accurate and high-quality data. These resources are not intended to provide a step-by-step guide to data collection.

Overarching Principles

Powering Health Equity Action with Online Data Tools: 10 Design Principles
PolicyLink & Ecotrust (2017)
https://nationalequityatlas.org/sites/default/files/10-Design-Principles-For-Online-Data-Tools.pdf
This document outlines ten principles to guide data tool creators in building tools that satisfy both research aims and strengthen community action toward health equity. This guide was developed for researchers, advocates, community members, planners, funders, and others interested in building, improving, or investing in equitable data tools.

Principles for Using Public Health Data to Drive Equity: A Guide to Embedding Equitable Practices Throughout the Data Life Cycle
CDC Foundation (nd)
https://www.cdcfoundation.org/data-equity-principles?inline
Geared towards program directors, survey developers, and analysts, this guide provides overarching guidance on how to embed equitable practices throughout the data life cycle: data planning, data collection, data access, data analysis, and data reporting and dissemination. The guide outlines five guiding principles and provides suggestions and actions for each stage of the data life cycle.

*Data Ethics Framework
General Services Administration (GSA) (nd)
https://resources.data.gov/assets/documents/fds-data-ethics-framework.pdf
The Framework’s purpose is to guide federal leaders and data users as they make ethical decisions when acquiring, managing, and using data to support their agency’s mission. The Framework does not include requirements or mandates of its own, but rather provides guidance in the form of tenets to encourage ethical decision making at all levels of the Federal Government.

Dabbling with data
Public Profit (nd)
DabblingintheDataGuidefromPublicProfit-1.pdf
This guidebook provides step-by-step instructions for fifteen different team-based data exercises to help people new to analyzing data understand basic data analysis concepts in a hands-on manner. These activities are an introduction to using data in a team environment and are most appropriate for individuals with little data analysis experience or in need of a refresher.

Culturally Responsive and Equitable Data Parties: A Method for Participatory Analysis and Sense-making in Virtual Spaces
The Administration for Children and Families (2023)
Culturally Responsive and Equitable Data Parties: A Method for Participatory Analysis and Sense-making in Virtual Spaces | The Administration for Children and Families (hhs.gov)
Data parties are fun events used to get buy-in from users and team members and to help users interpret and understand the data. The brief includes the following sections: (1) a definition of data parties, (2) strategies for designing and implementing virtual data parties, and (3) considerations for federal agencies to ensure the implementation of virtual data parties is culturally responsive and equitable. While the brief has an emphasis on virtual data parties, the recommendations are universal and can be applied to virtual, hybrid, and in-person data parties.

Data Party Planning Guide
University of North Carolina School of Medicine and the North Carolina Translational and Clinical Sciences Institute (2023)
TraCS CaSE program publishes Data Party Planning Guide (unc.edu)
This guide explains what data parties are, general advice for planning a data party, and an example data party including an agenda, logistics, party structure, and follow-up activities. In the appendix of the guide can be guide is an example data party agenda, example work sheets for various group activities, and example data party exit survey questions.

Considerations for Survey Data

Methods 101: Question Wording
Pew Research Center (2018)
https://www.youtube.com/watch?v=eFzGdQrr2K8&list=PLZ9z-Af5ISavJpPlvdMU4T-etIDOUmldk&index=6
This 5-minute video focuses on question wording. It covers avoiding jargon, defining key terms, and how much information to provide in questions. It explains and gives examples of how leading questions, double negatives, acquiescence bias, and context effects can influence responses. It closes with recommendations for question review and pretesting surveys.

Methods 101: Mode effects
Pew Research Center (2019)
https://www.youtube.com/watch?v=eRK_dXay5HY&list=PLZ9z-Af5ISavJpPlvdMU4T-etIDOUmldk&index=4
This 3-minute video discusses survey modes (online, phone, in-person, etc.) and how mode can affect responses. It discusses how interviewer involvement may influence responses to sensitive questions. It also touches on how survey mode and question order interact.

Best Practices for Survey Research
American Association for Public Opinion Research (AAPOR) (2022)
https://aapor.org/standards-and-ethics/best-practices/#1668111292345-ae04a1ba-5069
This document answers questions about planning surveys, designing a sample, designing the questionnaire, fielding a survey, and analyzing, and reporting on survey results.

*Q-Bank
Centers for Disease Control and Prevention (CDC) (nd)
https://wwwn.cdc.gov/qbank/Home.aspx
Q-Bank: Question Evaluation for Surveys (cdc.gov)
Q-Bank is a database of evaluated questions from federal surveys. Q-Bank links each survey question to reports including that question. Q-Bank can be searched by question topic (e.g., income, demographic, chronic health conditions), agency, and survey name. In addition, users can search for keywords. Q-Bank is intended to help users of survey data interpret the survey questions on which the data are based and understand the potential errors that might be associated with these questions. It may be helpful in deciding which questions to use to measure a topic of interest.

Considerations for Focus Groups

Designing and Conducting Focus Group Interviews
Richard A. Krueger (2002)
https://www.eiu.edu/ihec/Krueger-FocusGroupInterviews.pdf
This write-up provides a thorough review of focus group basics. The topics covered include skills needed in different roles, ideal candidates for focus groups, how to best phrase questions, suggestions on note taking, how to analyze results, and best practices for reporting out results.

Toolkit for Conducting Focus Groups
Community Toolbox, University of Kansas (nd)
https://ctb.ku.edu/sites/default/files/chapter_files/toolkitforconductingfocusgroups-omni.pdf
This toolkit walks readers through considerations in managing and facilitating focus groups from a practical perspective. The purpose of this toolkit is to offer considerations on how to build rapport with an audience, some steps in thinking through the planning of focus groups, tips for managing tricky situations that may arise during focus groups and outlines the roles and responsibilities of different members of the focus group facilitation team. It includes some templates for use.

Best Practices in Research and Evaluation: Focus Groups
ETR.org (nd)
https://www.etr.org/ebi/assets/File/etr_best_practices_focus_groups.pdf
This document provides basics on how to best deploy a focus group to support data collection for different initiatives such as supporting survey development or better interpreting the results of a large quantitative data collection. It further provides helpful suggestions on key steps in generating a focus group script, what to avoid and what to pursue. This guide is primarily geared toward in-person facilitation.

Considerations for Interviews

Conducting In-Depth Interviews: A Guide for Designing and Conducting In-Depth Interviews for Evaluation Input
Pathfinder International Tool Series (2006)
https://nyhealthfoundation.org/wp-content/uploads/2019/02/m_e_tool_series_indepth_interviews-1.pdf 
This guide outlines the advantages and limitations of using in-depth interviews for data collection in evaluation as well as providing a step-by-step process for how to plan for interviewing. Within the planning process the authors include tips for data collectors including those related to analysis and sharing results.

Best Practices in Research and Evaluation: Interviews
ETR.org (nd)
https://www.etr.org/ebi/assets/File/etr_best_practices_interviews.pdf
This document provides basics on how to use and make use of interviews for different data collection needs. The document outlines the advantages and disadvantages of using interviews and how interviews can support other forms of data collection. Interviews can be particularly useful when covering sensitive information that participants may not want to discuss in a focus group format.

*Conducting qualitative research with vulnerable groups
Canan Coskan (2023)
https://www.researchgate.net/publication/371158486_Conducting_qualitative_research_with_
vulnerable_groups
In this PowerPoint presentation is designed for experienced qualitative researchers. There are suggestions and citations for how to best approach in-depth interviews with individuals from vulnerable populations. The presentation outlines useful information such as a general definition of vulnerability, providing information on how to engage interviewees in meaningful ways, reviewing ethical considerations, and encouraging researchers to be reflective in their practice of data collection.

Considerations for Data Display

Advancing SAMHSA Programs in Reaching Equity (ASPIRE) Dashboard: A Case Study in Improving the Usability of Demographic Data
ASPE (2024)
ASPIRE Case Study
This content is in the process of Section 508 review. If you need immediate assistance accessing this content, please submit a request to Maura Reilly, maura.reilly@hhs.gov. Content will be updated pending the outcome of the Section 508 review.

This case study provides a general roadmap of important steps and decision points to navigate when building a useful data dashboard. Each section contains both the story of the Advancing SAMHSA Programs in Reaching Equity (ASPIRE) dashboard and suggested resource. This tool is useful for researchers and other scholars, policy makers, clinicians and institutions who want to improve the collection, analysis, and use of demographic data.

Asked and Answered: Visualizing Demographic Data
Evergreen (nd)
https://stephanieevergreen.com/visualizing-demographic-data/
This blog post provides concrete examples of ways to create effective data visualizations (e.g., charts) to represent different types of demographic data. It includes examples for representing data on gender as well as race.

*Privacy Preserving Data Visualizations
Avraam, D., Wilson, R., Butters, O. et al. Privacy preserving data visualizations. EPJ Data Sci. 10, 2 (2021).
https://epjdatascience.springeropen.com/articles/10.1140/epjds/s13688-020-00257-4#Sec0
This academic article presupposes that readers are familiar with statistical practices designed to preserve data privacy. It walks through three technical solutions for creating privacy preserving data visualizations: k-anonymization, deterministic anonymization, and probabilistic anonymization. These visualizations are beneficial when there is value to showing data points in a figure representing individual people, but those people may be identifiable, this is most frequently a challenge when visualizing data from small samples or unique populations.

Centering Accessibility in Data Visualization
Urban Institute (2022)
Do No Harm Guide Centering Accessibility in Data Visualization.pdf (urban.org)
Across ten chapters, this guide covers both aspects of incorporating accessibility when creating data visualizations (e.g., accounting for screen readers and adding alternative text) and non-technical aspects (e.g., designing for cognitive load) aspects of incorporating accessibility when creating data visualizations. Inside the guide are checklists and questions that can be used to guide the creation and review of accessible visualizations.

Do No Harm: Applying Equity Awareness in Data Visualization – A Check List
Urban Institute (2021)
https://www.urban.org/sites/default/files/2021/06/08/do-no-harm-guide-recommendations.pdf
Carelessly communicated data have an outsized capacity to mislead, misrepresent, and harm communities that already experience inequity and discrimination. This checklist presents general recommendations for how data practitioners can approach their work through a lens of diversity, equity, and inclusion and practice thoughtfulness as they work with and present their data. This guide can help data analysts and program managers, and program staff improve the quality data visualizations used for websites, posters and presentations. See also The Racial Equity In Data Visualization Checklist and Chapter 11: Practical Tactics for Gender Inclusivity in Data Reporting.

Data Visualization Handbook
Aalto University, Juuso Koponen and Jonatan Hilde‘n (2019)
Data Visualization Handbook (datavizhandbook.info)
This book on data visualization provides a comprehensive introduction to data visualization best practices. The authors cover the entire data visualization lifecycle, from initial idea and determining the audience, designing the visualization, typography and text, and creating overarching narratives through multiple data visualizations. This book provides deeper context for discussions on presenting data in an inclusive and equitable manner found in the other resources in this toolkit. Note: This link takes you to the free content and not the entire book. The site also offers a Visual Variables Sheet that describes which types of variables are best suited to different types of visualizations and a Workflow Model that can be helpful for defining and carrying out a data visualization project.

Visualise Data
Better Evaluation, Melbourne Australia (nd)
Visualise data - Rainbow Framework (betterevaluation.org)
This step in Better Evaluation’s Rainbow Framework walks readers through the process of selecting the most compelling graphics for their evaluation or analysis. For example, a stacked area chart is an appropriate format if one is trying to show how the composition of a group changes when information about the group’s composition is captured at multiple intervals over time and both relative and absolute differences in the composition of the group matter.

Inform, Engage, Inspire, with data visualization
Fresh Spectrum Data Visualization – Chris Lysy (2022)
Inform, Engage, Inspire, with data visualization. (freshspectrum.com)
This blog post uses example graphics from the New York Times to illustrate the importance of understanding the audience’s interest level in the topic when selecting data visualization strategies. The author breaks audiences down into three groups, people looking to be informed by a visualization because they already understand the topic, people who need to be engaged by the graphic because they are only mildly interested in the topic, and people with little to no interest in a topic who need a specific visualization to be more inspiring than informative.

*Open Questions about the Visualization of Sociodemographic Data
IEEE Workshop on Visualization for Social Good (VIS4Good), Melbourne, Australia (2023)
Open Question about the Visualization of Sociodemographic Data
Sociodemographic data is a common part of datasets related to people, including institutional censuses, health data systems, and human-resources files. This data is sensitive, and its collection, sharing, and analysis require careful consideration. Data visualization of sociodemographic data can reinforce stereotypes, marginalize groups, and lead to biased decision-making. It is, therefore, critical that these visualizations are created based on good, equitable design principles. This paper presents a set of open research questions around the visualization of sociodemographic data.

Executive Order 14075 (Advancing Equality for Lesbian, Gay, Bisexual, Transgender, and Intersex Individuals) directs agencies to address ongoing barriers that LGBTQI+ communities face in education, housing, the foster care system, access to health care, juvenile justice programs, and more. The EO recognizes that in order to advance equity for LGBTQI+ people, the Federal Government must continue to gather the evidence needed to understand the LGBTQI+ community, the barriers they face, and the policy changes the Federal Government can make to enable their health and well-being. The following are resources designed to Improve the quality of SOGI data. Also included are resources addressing concerns that adding SOGI questions to a data collection may harm the quality of the resulting data.

Sexual Orientation and Gender Identity (SOGI) Data Action Plan
HHS (2024)
https://www.hhs.gov/sites/default/files/hhs-sogi-data-action-plan.pdf
The HHS Sexual Orientation and Gender Identity (SOGI) Data Action Plan seeks to improve the collection of data related to the health and well-being of lesbian, gay, bisexual, transgender, queer, and intersex (LGBTQI+) people.

Guide for Using SWOT as a Planning Tool for SOGI Data Collection
ASPE (2024)
Guide for Using SWOT as a Planning Tool for SOGI Data Collection
This handout supports the data collection planning phase and offers users a strategy to think about the program/project’s existing strengths, weaknesses, opportunities and threats related to SOGI data. This resource is adapted from various SWOT analysis tools to tailor a short guide for LGBTQIA+ populations.

Sexual Orientation, Gender Identity, and Sex Development: Recommendations for Data Collection and Use in Clinical, Research, and Administrative Settings
Cheloff, A.Z., Jarvie, E., Tabaac, A.R., Katz-Wise, S.L., Fygetakis, L.M., & Keuroghlian, A, Harvard Medical School, Dean's LGBT Advisory Committee (2022)
https://dicp.hms.harvard.edu/sites/default/files/2022-10/SOGI%20Data%20Collection.pdf
When considering the use of sexual orientation and gender identity (SOGI) data in your work, there are many nuances in language, culture, generation, and individual identity that are important to consider when tailoring your approach. These guidelines will introduce some key considerations for collecting SOGI data.

Recommendations on Best Practices for SOGI Data Collection in Federal Statistical Surveys
The White House (2023)
https://www.whitehouse.gov/wp-content/uploads/2023/01/SOGI-Best-Practices.pdf
This resource is a comprehensive report published by the Office of the Chief Statistician of the United States on the current best practices for the collection of self-reported SOGI data on Federal statistical surveys. This resource helps users understand SOGI measurement approaches and offers considerations for including SOGI items on surveys. This resource also includes links to additional resources. It provides information to temper concern that adding SOGI questions to a survey will cause respondents to skip questions or abandon the survey altogether, harming the quality of the entire collection.

Updates on Terminology of Sexual Orientation and Gender Identity Survey Measures FCSM-20-03
Federal Committee on Statistical Methodology (2023)
https://nces.ed.gov/fcsm/pdf/fcsm_sogi_terminology_fy20_report_final.pdf
This report provides a comprehensive overview of recent research about how LGBTQI+ individuals situate themselves into response options that may become part of a survey tool. It provides evidence-based insights identified from the literature about response option terminology and associated methodological implications.

Evaluations of Sexual Orientation and Gender Identity Survey Measures: What Have We Learned?
Federal Interagency Working Group on Improving Measurement of Sexual Orientation and Gender Identity in Federal Surveys (2016)
https://dpcpsi.nih.gov/sites/default/files/Evaluations_of_SOGI_Questions_20160923_508.pdf
This older resource summarizes lessons learned from the collection of SOGI data to date and discusses the rigor of a set of measures selected to allow users to understand measurement properties for the population of interest, and to determine its fit for its intended purpose. This resource can help users identify ways in which SOGI has been measured and learn from previous implementation of these approaches in the Federal Government. In general, sexual identity items tested in the studies reviewed performed well, and most respondents appeared to have little difficulty answering the items. Some research did identify differences in responses/estimates, depending on how the item was worded and collected.

*Are Sexual Minorities Hard-to-Survey? Insights from the 2020 Census Barriers, Attitudes, and Motivators Study (CBAMS) Survey
Journal of Official Statistics, Vol. 35, No. 4, pp. 709–729 (2019)
http://dx.doi.org/10.2478/JOS-2019-0030
Using a nationally representative survey that included sexual orientation (the Census Barriers, Attitudes, and Motivators Survey), the researchers examined level of effort, the Census Bureau’s Low Response Score (LRS), and stated intent to respond to the 2020 Census as proxy measures to explore this assumption. There was no evidence that respondents that identifies as lesbian, gay, and bisexual (LGB) required higher levels of effort to secure participation in the survey. Additionally, compared to straight respondents, lesbians, gays, and bisexuals had a higher intent to respond to the 2020 Census.

Sexual Orientation, Gender Identity, and Sex Characteristics Subcommittee
Federal Committee on Statistical Methodology (nd)
https://www.fcsm.gov/groups/sogisc/
This website provides links to research and recommendations for the collection of data on sexual orientation, gender identity, and sex characteristics. The Sexual Orientation, Gender Identity, and Sex Characteristics (SOGISC) Subcommittee promotes coordination between federal agencies on issues relating to measuring sexual orientation and gender identity in federal data collections.

Measuring Sex, Gender Identity, and Sexual Orientation
National Academies of Sciences, Engineering, and Medicine (2022)
https://nap.nationalacademies.org/catalog/26424/measuring-sex-gender-identity-and-sexual-orientation
Sex, gender identity, and sexual orientation are key indicators of the demographic diversity in the United States. Sex and gender are often conflated under the assumptions that they are mutually determined and do not differ from each other; however, the growing visibility of transgender and intersex populations, as well as efforts to improve the measurement of sex and gender across many scientific fields, has demonstrated the need to reconsider how sex, gender, and the relationship between them are conceptualized. The report recommends standardized language to be used in survey questions that ask about a respondent's sex, gender identity, and sexual orientation.

*Interagency Technical Working Group on Sexual Orientation and Gender Identity Items in the Household Pulse Survey: Report and Recommendations
Office of Management and Budget (2021)
https://omb.report/icr/202106-0607-003/doc/112605500
The Household Pulse Survey (HPS), conducted by the U.S. Census Bureau, is designed to provide researchers and decision-makers with timely, relevant, and accurate information on the coronavirus pandemic’s impact on the American public. Survey content on the HPS is provided by several different agency partners to reflect priority information needs across government.

Federal Evidence Agenda on LGBTQI+ Equality
National Science and Technology Council (2023)
https://www.whitehouse.gov/wp-content/uploads/2023/01/Federal-Evidence-Agenda-on-LGBTQI-Equity.pdf
This report establishes the Evidence Agenda described in EO 14075 and provides a roadmap for federal agencies as they work to create their own data-driven and measurable SOGI Data Action Plans to help assess, improve, and monitor the health and well-being of LGBTQI+ people over time.

*Strategies to Improve Measurement of Sexual Orientation and Gender Identity Among Youth
Journal of Adolescent Health 71 (2022)
https://www.jahonline.org/article/S1054-139X(22)00646-2/fulltext
This article provides strategies that can help researchers mitigate challenges for collecting sexual orientation and gender identity (SOGI) data with youth.

Podcast Series Collecting Data on Sexual Orientation, Gender Identity, and Expression in Child Welfare
James Bell (nd)
https://www.acf.hhs.gov/sites/default/files/documents/cb/SOGIE_podcast_flyer.pdf
This podcast series is produced and distributed by James Bell Associates in collaboration with the National SOGIE Center on behalf of the Children’s Bureau, Administration for Children and Families, U.S. Department of Health and Human Services, under contract number 47QRAA20D0008. This site links to four podcast episodes: Episode 1: Understanding the Importance of SOGIE Data; Episode 2: Implementing Organizational Change; Episode 3: Storytelling With SOGIE Data; and Episode 4: Moving Beyond Data Collection.

Sexual Orientation and Gender Identity Questions: Information for Patients
National LGBTQIA+ Health Education Center (nd)
https://www.lgbtqiahealtheducation.org/publication/sexual-orientation-and-gender-identity-questions-information-for-patients/
This pamphlet on sexual orientation and gender identity (SOGI) questions can be handed out to patients in waiting rooms or exam rooms. The pamphlet explains: why SOGI data are being collected, what each SOGI term means, and how the information will be kept private. The pamphlet comes in several languages.

Ready, Set, Go! Guidelines and Tips For Collecting Patient Data on Sexual Orientation and Gender Identity (SOGI) – 2022 Update
National LGBTQIA+ Health Education Center (2022)
https://www.lgbtqiahealtheducation.org/publication/ready-set-go-a-guide-for-collecting-data-on-sexual-orientation-and-gender-identity-2022-update/
This guide was designed to help health centers and other health care organizations successfully collect sexual orientation and gender identity (SOGI) data and document the data into the electronic health record (EHR). This publication was supported by the Health Resources and Services Administration (HRSA) of the U.S. Department of Health and Human Services (HHS) as part of the award U30CS22742. See also Cultural Adaptation of Measures and Tools for Sexual Orientation and Gender Identity (SOGI) Data Collection.

Cultural Adaptation of Measures and Tools for Sexual Orientation and Gender Identity (SOGI) Data Collection
National LGBTQIA+ Health Education Center (nd)
https://www.lgbtqiahealtheducation.org/publication/cultural-adaptation-of-measures-and-tools-for-sexual-orientation-and-gender-identity-sogi-data-collection/
This publication was designed as a companion resource to Ready, Set, Go! Guidelines and Tips for Collecting Patient Data on Sexual Orientation and Gender Identity (SOGI) to explain the importance of SOGI data collection to identify and address the unique needs of sexual and gender minority patient populations, with special consideration for culturally and linguistically diverse patients who are part of LGBTQIA+ communities.

*Data Collection and the Paperwork Reduction Act– SOGI Data Checklist For HHS Operating and Staff Divisions
HHS (2024)
Data Collection and the Paperwork Reduction Act– SOGI Data Checklist For HHS Operating and Staff Divisions
This document is intended for HHS staff or contractors responsible for Federal data collections that are subject to the Paperwork Reduction Act. It provides a brief background related to requirements for the collection of sound SOGI data, a process for determining if SOGI data elements are required for the specific data collection, guidance for requesting a modification for previously cleared collections, and links to relevant resources.

*Eleven years of gender data visualization: A step towards more inclusive gender representation – Video from Author
IEEE Visualization Conference (2023)
Eleven Years of Gender Data Visualization: Towards more Inclusive Gender Representation | VIS 2023 (youtube.com)
This ten-minute video provides an overview of the academic article of the same name along with additional visualizations and advice from the authors. The video and article provide an overview of how women are most often depicted in data visualizations and how that shapes understanding of the data behind the visualizations.

Executive Order 13985 (Advancing Racial Equity and Support for Underserved Communities Through the Federal Government) notes that many Federal datasets are not disaggregated by race and/or ethnicity. This lack of data has cascading effects and impedes efforts to measure and advance equity. A first step to promoting equity in Government action is to gather the data necessary to inform that effort. The resources in this section provide guidance focused on the collection and use of high-quality data on race and ethnicity.

Planning for Data Collection

Statistical Policy Directive No. 15: Standards for Maintaining, Collecting, and Presenting Federal Data on Race and Ethnicity
Office of Management and Budget (OMB) (2024)
https://www.federalregister.gov/public-inspection/2024-06469/statistical-policy-directive-no-15-standards-for-maintaining-collecting-and-presenting-federal-data
The revised standards presented in the Notice adopt several revisions intended to improve the quality and usefulness of Federal race and ethnicity data. The provisions of these standards were effective March 28, 2024 for federal record keeping or reporting requirements that include racial or ethnic information.

Federal Committee on Statistical Methodology (FCSM) Equitable Data Toolkit
Federal Committee on Statistical Methodology (2023)
https://nces.ed.gov/fcsm/edt/index.html
Influenced by Executive Order 13985, the Federal Committee on Statistical Methodology (FCSM) developed the FCSM Equitable Data Toolkit (Toolkit) to provide federal agencies with tools that support equity analyses with a focus on historically underserved populations. The Toolkit provides information and resources for Statistical Officials, Chief Data Officers, agency staff, and practitioners seeking guidance for: 1) Refining wording of survey items to operationalize concepts that facilitate identification or self-identification of characteristics in ways that are relevant for equity analysis. 2) Overcoming sample-size issues that affect the precision of estimates for population groups that are small in number, geographically diffuse, and possibly difficult to identify or self-identify. 3) Linking across data sources to build information including matching strategies and evaluation of linkage success. 4) Addressing data-protection issues when a survey participant is a member of a relatively small population group and one of relatively few such members included in the sample; these issues are especially salient when data are examined for subgroups defined by a combination of variables (race/ethnicity, geographic location, etc.) that can result in a number of observations in the subgroup that is too small for drawing statistical conclusions. 5) Issues of definition and measurement involving geography and persistent poverty.

Data Analysis

A Resource Guide for Using Medicare’s Enrollment Race and Ethnicity Data
US Department of Health and Human Services – Office of Inspector General (2023)
https://oig.hhs.gov/oei/reports/OEI-02-21-00101.pdf
This guide provides a concise, user-friendly explanation of the origins and limitations of Medicare's enrollment race and ethnicity data and offers considerations for the use of these data. The guide is based on the findings from a June 2022 report, Inaccuracies in Medicare’s Race and Ethnicity Data Hinder the Ability to Assess Health Disparities.

A Toolkit for Centering Racial Equity Throughout Data Integration
Actionable Intelligence for Social Policy (AISP), University of Pennsylvania (2020)
https://aisp.upenn.edu/centering-equity/
This toolkit describes positive and problematic practices for centering racial equity across the six stages of the data life cycle. This toolkit can be used by data analysts to better understand how to analyze their data. The toolkit highlights positive and problematic practices as well as case studies where key topics are demonstrated. For the specific practices and recommendations regarding algorithms and statistical tools, refer to the chapter “Racial Equity in Algorithms/Statistical Tools” (pages 25-28). Additionally, for resources regarding algorithms for race & ethnicity data use, see pages 72-73.

Disaggregating Race and Ethnicity Data

Best Practices for the Disaggregation of Federal Data on Asian Americans and Pacific Islanders
White House Initiative on Asian Americans and Pacific Islanders Interagency Working Group Data and Research Subcommittee (2016)
https://nces.ed.gov/fCSM/pdf/WHIAAPI_2016_spread_508c.pdf
To provide more accurate and meaningful information on the Asian Americans and Pacific Islander (AAPI) community to policy makers, the press, and the public, federal statistical agencies should attempt to provide disaggregated AAPI data to the extent possible. To explore the challenges of this issue and potential solutions, the White House Initiative on Asian Americans and Pacific Islanders Interagency Working Group established a subcommittee of key federal agencies to discuss data disaggregation. The Data and Research Subcommittee developed this document, which contains an overview of best practices for providing disaggregated AAPI data.

Disaggregating Data to Measure Racial Disparities in COVID-19 Outcomes and Guide Community Response — Hawaii, March 1, 2020–February 28, 2021
Centers for Disease Control and Prevention: Morbidity and Mortality Weekly Report (2021)
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8445382/pdf/mm7037a1.pdf
This publication describes a specific approach to analyzing data from a population in which a large proportion identifies as multiracial. Analyses were conducted with groups that were not mutually exclusive, including persons of one race alone or in combination with one or more races. Using this approach, persons of more than one race were counted multiple times, depending upon the number of race groups recorded.

Identifying and Assessing Critical Gaps in the Data

A Framework for Stratifying Race, Ethnicity, and Language Data
Health Research & Educational Trust in partnership with the American Hospital Association (2014)
https://www.aha.org/system/files/2018-01/framework-race-ethnicity-language-data-2014.pdf
This systematic five-step stratification framework is designed to assist hospitals and other large complex organizations in pinpointing healthcare disparities within patient data. Emphasis is placed on the creation of dashboards to enhance the reporting of stratified data's impact. Utilizing this framework can help program directors, analysts, and program staff use data to gain valuable insights into disparities and work towards achieving health equity.

Addressing Gaps in Public Health Reporting of Race and Ethnicity Data for COVID-19: Findings & Recommendations Among 45 State & Local Health Departments
Council of State and Territorial Epidemiologists (2022)
https://preparedness.cste.org/wp-content/uploads/2022/04/RaceEthnicityData_FINAL.pdf
The Council of State and Territorial Epidemiologists (CSTE) conducted an assessment to identify the factors that are impacting the completeness and quality of race and ethnicity data for COVID-19 at public health agencies, as well as the solutions that may help mitigate these limiting factors . The assessment was completed by State Epidemiologists and designated staff at state, territorial, local, and tribal public health agencies.

Reporting the Data

Applying Racial Equity Awareness in Data Visualization
The Urban Institute (2020)
https://www.urban.org/research/publication/do-no-harm-guide-applying-equity-awareness-data-visualization
This brief article walks through actionable ways to apply the values of equity and inclusivity to the data visualization process. The article contains visual examples and suggestions for best practices.

Are your Data Visualizations Racist?
Stanford Social Innovation Review (2021)
Data Visualizations and Racial Equity: Three Best Practices and Strategies (ssir.org)
This blog post, created by the authors of the Do No Harm Guide, lists three things to keep in mind when creating equitable and inclusive data visualizations: use inclusive language in and to describe the visualization, consider about how data are ordered in the visualization, and choose colors thoughtfully.

*Recommendations on the use and reporting of race, ethnicity, and ancestry in genetic research: Experiences from the NHLBI TOPMed program
Cell Genomics (2022)
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9481067/pdf/main.pdf
Researchers with the NHLBI Trans-Omics for Precision Medicine (TOPMed) program offer commentary and recommendations on the use of race, ethnicity, and ancestry across the arc of genetic research, including data harmonization, analysis, and reporting. These recommendations are applicable to basic and translational genomic research in diverse populations with genome-wide data.

Reporting Style – AMA Guidance for Reporting Race and Ethnicity in Medical and Science Journals
American Medical Association (2023)
https://jamanetwork.com/journals/jama/fullarticle/2783090 This guidance from the AMA Manual of Style Committee provides updated recommendations and suggestions that encourage fairness, equity, consistency, and clarity in use and reporting of race and ethnicity in medical and science journals.

Reporting Style – APA Style Journal Article Reporting Standards (APA-JARS)
American Psychological Association (2023)
https://apastyle.apa.org/blog/race-ethnicity-culture-reporting-standards
APA Style Journal Article Reporting Standards (APA Style JARS) are a set of standards designed for journal authors, reviewers, and editors to enhance scientific rigor in peer-reviewed journal articles. The Journal Article Reporting Standards for Race, Ethnicity, and Culture (JARS–REC) are new standards within APA Style JARS that address the discussion of race, ethnicity, and culture in scientific manuscripts.